With the emergence of the Internet, large amounts of information can be shared and distributed between any number of interconnected users. The users can be remotely located, spanning multiple continents. Typically, the users store information in data files (“files”). Many user applications such as multimedia applications running on computing systems distribute large files (e.g., multimedia files), which can exceed one gigabyte in memory space. Distributing large files between user applications over a network such as the Internet can be problematic.
For instance, many Internet applications use the Transfer Control Protocol/Internet Protocol (TCP/IP) layers to send files separated as packets over the Internet. The IP layer handles the actual delivery of the packets and the TCP layer ensures that each packet is delivered and reordered correctly for its destination application. To deliver packets, TCP establishes a connection between two TCP endpoints, defined by an IP address and a port number. An IP address identifies a computing system and the port number identifies an application operating within that computing system. Each packet contains a sequence number. The sequence numbers are used to acknowledge received packets and to reorder correctly packets at a receiving end in the case of packets being received out of order.
To ensure reliable delivery of packets, TCP must receive acknowledgement that delivered packets were received at a receiving end. In this process, TCP uses a “sliding window” algorithm to dynamically calculate the maximum number of unacknowledged (in-transit) packets to allow before enacting flow control (preventing further sends). The sliding window algorithm is designed to prevent congestion while still allowing the window to grow large enough to accommodate fast link speeds. Unfortunately, the algorithm often treats latency induced by sending packets large distances and latency induced by actual congestion similarly as it is programmatically difficult to make a distinction between the two at the level on which TCP operates.
In particular, If a TCP connection experiences high latency, TCP assumes congestion in which case TCP decreases the size of the “window.” Furthermore, TCP may also resend the packets if not acknowledged within a certain period of time. However, in many instances, the TCP connection is over a high speed connection line, but the receiving end is at a remote location, which can cause an inherent latency in the delivery and acknowledgement of packets. For example, an 8 mbps connection line used for sending packets to a remote user will experience latency at the receiving end that causes the overall throughput to be a small fraction of the maximum possible due to the congestion control mechanism of TCP. As such, applications may not be able to utilize the full available bandwidth on a connection line when sending large files.
Another limitation with delivering large files over high-speed connections is that the network throughput can exceed the file input/output (I/O) capabilities in the sending and receiving computing systems. For instance, multiple memory devices may be required to store and buffer a large file. If multiple memory devices are required to store or buffer a large file, to seek the appropriate segments of the file or location of the segment in memory can be time consuming. Such file I/O processes can thus limit the throughput on a high speed connection line. Therefore, delivering large files at the maximum possible speed requires efficient file I/O processes.
There exists, therefore, a need for an improved method and system that overcome the limitations of transferring data files.